class: center, middle, inverse, title-slide .title[ # Systematic Literature Review ] .subtitle[ ## Part 1: Introduction ] .author[ ### Faisal Mustafa ] .date[ ### 2024-05-28 ] --- <h1> Outline of Part 1:</h1> <hr> <h2> This part of presentation covers:</h2> <ul> <li> Definition </li> <li> Research questions </li> <li> Steps of SLR research </li> <li> Writing an SLR paper </li> </ul> Slides are accessible at https://faisalmustafa.github.io/SLR/ <img class = "qr" src="images/qr-code.png" width = 300px alt="QR code of "> --- <h1> Definition </h1><hr class="vspace"> <h7> <em>A method of making sense of large bodies of information, and a means to contributing to the answers to questions about what works and what does not.</em></h7> <br><br> <table style="width:70%"> <tr> <th>Narrative LR</th> <th>Systematic LR</th> </tr> <tr> <td>Variety of styles</td> <td>Structured approach</td> </tr> <tr> <td>No defined method</td> <td>Rigorous method</td> </tr> <tr> <td>No specified analysis</td> <td>Synthesis to answer RQ / achieve research objectives</td> </tr> </table> <p class = "citation"> Jesson, J. K., Matheson, L., & Lacey, F. M. (2011). <em>Doing your systematic literature review: Traditional and systematic technique.</em> SAGE Publications Ltd,</p> --- <h1> Publishable SLR: <span class="subtitle"> To stand out of the review </span></h1><hr> -- ⛯ The objective of SLR: To answer the research question.<br><br> -- ⛯ Start SLR with research question: Be as specific as possible. <br>   Example 1 = https://doi.org/10.3389/fpsyg.2022.1075818 <br>   Example 2 = https://doi.org/10.1080/17501229.2024.2355296 <br><br> -- ⛯ Better format: <img src="images/SLR format.png" width = 75% alt="My SLR format">   Example 1 = https://doi.org/10.1016/j.ijer.2024.102380 <br>   Example 2 = https://doi.org/10.1080/17501229.2024.2355296 --- <h1>Steps in doing SLR</h1><hr> Step 1: Define the real objective (for publication) <br> Step 2: Decide research question <br> Step 3: Define inclusion and exclusion criteria <br> Step 4: Search the literature <br> Step 5: Screen the papers <br> Step 6: Retrieve full papers <br> Step 7: Conduct quality assessment <br> Step 8: Coding process <br> Step 9: Synthesis to achieve the research objective <br> Step 10: Writing up the SLR paper --- <h1> Real objective and RQ </h1><hr> <h2>My scenario:</h2> <span class="highlight"> Topic:</span> Innovation in digital transformation [scope = in Southeast Asian higher education] <br><br> <span class="highlight"> Gap:</span> But no clear definition of innovation <br><br> <span class="highlight">Objective:</span> Criteria of innovation <br><br> <span class="highlight"> Research questions:</span> What are the types of innovation in digital transformation in Southeast Asian higher education? <footer> Any questions so far? </footer> --- <h1> Inclusion criteria and literature search</h1><hr> <ol> <li> Databases: Scopus, Webs of Science, ProQuest, etc.</li> -- <li> Publication type: Peer reviewed such as journal articles and conference proceedings </li> -- <li> Period. Be clear of the reason </li> -- <li> Keywords: Teach yourselves Boolean AND, OR, AND NOT <br> Let's demonstrate at https://www.scopus.com, https://www.webofscience.com/wos/, and https://www.proquest.com </li> <li> Including only open-access articles is not recommended </li> </ol> <em> Insert a figure here </em> <footer> Any questions so far? </footer> --- <h1> Lit. preparation, screening and retrieval</h1><hr> <footer> lit. = literature </footer> If excel is more preferredf for final list preparation: <ul> <li> Sorting </li> <li> Conditional formatting </li> </ul> -- Screening process: <ul> <li> Read title, abstract, and keywords (but mostly abstract) </li> <li> Check type of publication because the database search might be incorrect </li> <li> Record reasons of exclusion </li> </ul> -- The full text of the final list can be obtained from: <ul> <li class = "highlight"> University library subscription </li> <li class = "highlight"> <em> Perpusnas</em> subscription </li> <li> Researchgate or other free repository </li> <li> Authors </li> </ul> <h8>All process and the number of publications resulted after each process must be recorded.</h8> --- <h1> Quality assessment:<span class="subtitle"> Quality appraisal of RAs</span></h1><hr> <ul class = "no-tab"> <li> To ensure that the coding process is smooth </li> <li> High quality articles provide all information we need for SLR <li> low quality papers mush be excluded from the review </ul> <h8> Framework for quality appraisal of research articles (RAs) (Nguyen, et al., 2024) </h8> <table class="blueTable"> <thead> <tr> <th style="width:87%">Criteria</th> <th style="width:13%; text-align: left;">Yes (1)   No (0)</th> </tr> </thead> <tbody> <tr> <td>1. Are the research questions (RQs) or objectives clearly and appropriately defined?</td><td></td></tr> <tr> <td>2. Is the research design appropriate for addressing the RQ(s)/objective(s) and clearly presented in the article?</td><td></td></tr> <tr> <td>3. Is the sampling strategy appropriately justified?</td><td></td></tr> <tr> <td>4. Does the article clearly describe the setting of data collection?</td><td></td></tr> <tr> <td>5. Is/Are the method(s) of data collection appropriate for addressing the RQ(s)/objective(s) and clearly presented in the article?</td><td></td></tr> <tr> <td>6. Is/Are the key concept(s) (e.g. PLC) clearly defined in the article?</td><td></td></tr> <tr> <td>7. Is/Are the method(s) of data analysis appropriate for addressing the RQ(s)/objective(s) and clearly presented?</td><td></td></tr> <tr> <td>8. Is/Are the research question(s) or objective(s) answered?</td><td></td></tr> <tr> <td>9. Are the discussion/conclusion(s)/implication(s) data appropriate?</td><td></td></tr> <tr> <td>10. Is there evidence of attention to ethical issues?</td><td></td></tr> <tr> <td class="bold">Total Score</td><td></td></tr> </tbody> </tr> </table> <p style="font-size: 17px"> <strong>Yes</strong> = Score 1; <strong>No</strong> = Score 0; <strong>High quality</strong> = 8-10; <strong>Medium quality</strong> = 5-7; <strong>Low quality</strong> = 0-4 </p> <p class="citation"> Nguyen, D., Boeren, E., Maitra, S., & Cabus, S. (2024). A review of the empirical research literature on PLCs for teachers in the Global South: Evidence, implications, and directions. <em>Professional Development in Education, 50</em>(1), 91–107. <a href="https://doi.org/10.1080/19415257.2023.2238728">https://doi.org/10.1080/19415257.2023.2238728</a> </p> --- <h1> Coding process </h1><hr> <h3> Phase A: Coding preparation </h3> <strong>Step 1:</strong> Determine framework <br> <strong>Step 2:</strong> Design a coding protocol <br> <span style="margin-left: 67px"> ⬥ Prioritize <em> Discussion → Conclusion → Results </em></span><br> <span style="margin-left: 67px"> ⬥ Code only one sentence or specific part of the sentence</span><br> <span style="margin-left: 67px"> ⬥ What sentence to code when the idea is expressed in one paragraph</span><br> <span style="margin-left: 67px"> ⬥ Etc.</span><br> <strong>Step 3:</strong> Recruit another coder (or more coders, depending on the size of the project) <br> <strong>Step 4:</strong> Socialize the framework and coding protocol to other coders --- <h1> Coding process </h1><hr> <h3> Phase B: Inter-coder reliability</h3> <img src="images/ICR.png" width = 73% alt="Inter-coder reliability"> <div class="topright"> <table class="blueTable" style ="width:25%"> <thead> <tr> <th style="width:10%; text-align: center;">Kappa statistic</th> <th style="width:15%; text-align: center;">Strength of Agreement</th> </tr> </thead> <tbody> <tr> <td style ="text-align:center"> <0.00 </td><td style ="text-align:center">Poor</td></tr> <tr> <td style ="text-align:center">0.00 - 0.20</td><td style ="text-align:center">Slight</td></tr> <tr> <td style ="text-align:center">0.21 - 0.40</td><td style ="text-align:center">Fair</td></tr> <tr> <td style ="text-align:center">0.41 - 0.60</td><td style ="text-align:center">Moderate</td></tr> <tr> <td style ="text-align:center">0.61 - 0.80</td><td style ="text-align:center">Substantial</td></tr> <tr> <td style ="text-align:center">0.81 - 1.00</td><td style ="text-align:center">Almost perfect</td></tr> </tbody> </table> </div> <p class="citation"> Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. <em>Biometrics, 33</em>(1), 159-174. <a href="https://doi.org/10.2307/2529310"> https://doi.org/10.2307/2529310 </a></p> --- <h4> Systematic Literature Review </h4> <h5> Part 2: Inter-coder Reliability </h5> <h6> Faisal Mustafa </h6> <h6> 2024-05-28 </h6>